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Abstract 

An integer polynomial p oi n variables is called a threshold gate for 
the Boolean function f oi n variables if for all x G {0, 1}" f{x) = 1 if 
and only if p{x) ^ 0. The weight of a threshold gate is the sum of its 
absolute values. 

In this paper we study how large weight might be needed if we fix 
some function and some threshold degree. We prove 2^^^ ^ '> lower 
bound on this value. The best previous bound was 2^^^"'''*) |12) . 

In addition we present substantially simpler proof of the weaker 
20(2 / ) ^Q^gj. bound. This proof is conceptually similar to other proofs 
of the bounds on weights of nonlinear threshold gates, but avoids a lot 
of technical details arising in other proofs. We hope that this proof 
will help to show the ideas behind the construction used to prove these 
lower boundfl 

1 Introduction 

Let /: {0,1}" {0,1} be a Boolean function. A threshold gate for the 
Boolean function / is an integer polynomial ^(x) of n variables x = {xi, . . . , x„) 
such that for any x E {0, 1}" we have f{x) = 1 if and only if p(x) ^ 0. In 
other words, for all x € {0,1}" it is true that f{x) = sgnp(x), where we 
adopt the following definition of the sign function: sgn(t) = I if t ^ and 
sgn(t) = otherwise. 
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Thus, threshold gates are just representations of Boolean functions as 
the signs of polynomials. The formal study of such representations started 
in 1968 with the seminal monograph of Minsky and Papert [7]. Since then 
representations of this form found a lot of applications in circuit complexity, 
structural complexity, learning theory and communication complexity (see, 
for example [HI [3 [21 [H] ) . 

Two key complexity measures of threshold gates are the degree of the 
threshold gate and its weight. The degree degp of a threshold gate p is just 
the degree of the polynomial. The weight W{p) of a threshold gate p is the 
sum of absolute values of all its coefficients. 

The complexity measures of a Boolean function / related to these com- 
plexity measures of threshold gates are the minimal threshold degree of a 
threshold gate for / which we denote by deg_|_ / and call the threshold degree 
and the minimal weight of a threshold gate for /. Both of these complexity 
measures play an important role in theoretical computer science (see the 
references above). In this paper we are interested in the minimal possible 
value of the weight of a threshold gate for some function / when the degree 
of the threshold gate is bounded. It is convenient to denote by W{f, d) the 
minimal weight of a threshold gate of degree at most d for /. Note that this 
value is defined only if d ^ deg^ /. It is also not hard to see that for all / 
we have deg_|_ / ^ n and W{f,n) ^ 2^^'^^ (just consider the polynomial p 
such that p{x) = f{x) for all x). 

The first results on the value of W{f, d) were proven for d = 1. In [9j (see 
also [8] and [4j ) it was proven that for all / with deg^ / = 1 it is true that 
W{f, 1) = n'-^^"'\ For a long time only lower bounds of the form W{f, 1) = 
2^(") were known (see pjQ] for one of early results). Tight lower bound was 
proven in [4J , that is the function / with deg_|_ / = 1 was constructed such 



Concerning higher degree d, upper bound can be easily extended from 
the case d = 1. Namely for all / with deg^ f ^ d it is true (and easy to see) 
that W{f, d) = n'^idr^'') (see [I5l[3l[l2]). Note that this upper bound is much 
worse than for the case d = 1. As for the lower bound, the one which is better 
than for the linear case (and also tight for constant d) appeared in ^12j. For 
any constant d the function / of threshold degree d was constructed there 
such that W{f, d) = n^^" ^ (constant in here depends on d). It is implicit 
in that paper though that the argument works for nonconstant d also and 
the resulting lower bound (with the dependence on d) is 



that W{f, 1) = n' 



n(n) 




(1) 
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For this result another proof was given in [T] . 

Thus it turns out that the required weight grows with the growth of 
degree d. In this paper we are interested how large it might grow (note that 
for d = n the weight is small again: VF(/, n) ^ 2'^(")). That is we study the 
value 

W = max max W{f,d). 

d f: deg±(/)>d 

The lower bound ([T|) works even for d linearly depending on n and so gives 
doubly exponential lower bound on this value. But it works only for d ^ 
(n— c)/32, where c is some constant, so the best lower bound we get from [12] 

is 2^(2"''^). 

In this paper we prove the following bound. 
Theorem 1. W ^ 2^(22"/5) 

We note that the best upper bound known is simple 2'^^"^") (this can be 
deduced from the upper bound for the case d = 1 and the fact that there 
are at most 2" monomials). 

To prove our lower bound we adopt the strategy of [12] and provide a 
unified treatment of the argument of that paper. In short, the proof strategy 
is as follows. Starting from some function of threshold degree 1 (with some 
additional properties) that requires large weight when represented by degree- 
1 threshold gates we construct its "d-dimensional" generalization in a very 
specific way. For this generalization we are able to prove lower bounds for 
degree-d threshold gates and due to the specific features of our generalization 
we can prove strong lower bound. 

In the paper [12j the construction of the function starts with the function 
constructed by Hastad in [4] to prove the optimal lower bound for the case 
d = 1. This helps to get n in the base of the exponent in the lower bound and 
thus to prove strong lower bound for the case of constant d. On the other 
hand, Hastad's function is very complicated and have desired properties only 
for large enough number of variables (16 variables). This does not allow us 
to prove lower bound for d close to n. In this paper we start with a much 
simpler functions having required properties starting from just 3 variables. 
With this function we cannot get n in the base of the exponent, but on the 
other hand we are able now to prove bounds for much larger d and thus get 
better lower bound on W. 

We start exposition of our result by giving a simpler proof of the weaker 
bound of 2^(2"''*) . In this proof we are able to avoid a lot of technical 
complications arising in the proof of [12j and make the function for which 
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we prove the bound much simpler (here we use as a starting function of 
threshold degree 1 well known "greater than" function). We hope that 
this makes the proof easier to read and helps to show the ideas behind the 
construction which were not very clear in |12j . 

After that we define another starting function and explain how to change 
the proof to get W ^ 2^*^^^"'^^) lower bound. The idea here is not only that we 
can prove the bound for larger d, but also that, roughly speaking, choosing 
the good function we can remove the constant 2 from the denominator of 
the term (^) in the exponent in the bound ([I]). 

Besides representation of Boolean functions as /: {0, 1}" {0, 1}, also 
representation of the form /: {—1, +1}" {—^, +1} turns out to be useful 
in complexity theory. For this representation we can also consider threshold 
gates and also define corresponding measures of the functions. Note that 
we can switch from one representation to another one by a simple linear 
transform. Thus the threshold degree of the function does not depend on 
the representation and the threshold weight may change only by 2" multi- 
plicative factor (see [6j for more information on relations between threshold 
weights in these two settings). Since this factor is very small compared to 
our lower bound, our result is true for both representations and in the proof 
we can choose the one of two presentations of Boolean functions which is 
more convenient to us. For the proof of weaker bound we will use {0, 1} 
variables and for the stronger one — {—1, 1} variables. 

We note in the conclusion that if we have the lower bound S on the 
minimal weight for the function of n variables and for degree d, it is easy 
to translate it to exactly the same bound 5 for n' = n + c and d' = d + c 
for any c, even depending on n (see, for example, |13j . Corollary 1). This 
observation allows us to deduce strong lower bounds on weights of threshold 
gates of degree close to n. 

Theorem 2. For any e > and d ^ {1 — e)n there is an explicit function f 
such that W{f, d) = 2^" " . For any d ^ n — 2(1 + e) log n there is an explicit 
function f such that W{f,d) = 2^^'^^^"\ 

The rest of the paper is devoted to the formulation and the proof of 
our results. In Sections [2] and [3] we give a simple proof for the weaker 
bound: in the former we construct the function for which in the latter 
we prove the lower bound. In Section H] we explain how to change the 
proof to give the stronger bound. 
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2 Construction of the Function 



In this section we present the construction of the function for which we 
prove a weaker form of our bound. Our function is the generahzation of GT 
function. 

Definition 3. For Boolean € {0, 1}" let GT{x,y) = 1 iff x ^ y, where 
X = (xi, . . . , Xn) and y = {yi, . . . , yn) are considered as binary representa- 
tions of integers with x„ and y„ being the most significant bits. 

Our function will depend on m = 2n variables 

(x, y) = (xi, . . . , x„, yi, . . . , yn) G {0, 1}™. 

Let us fix some ki, . . . ,ka such that Yli=i ki = m and partition the input 
variables x and y in d groups of size ki, . . . ,kd, that is 

{x,y) = (x\x2,...,x'^,y\y^...,?/'^), 

where x*,?/* G {0, for all i. 

Let us denote by [k] the set {1, . . . , A;}. Let us denote by <i the following 
ordering of the set [k]: 1, 2, 3, . . . , /c — 1, k, and by <o the reverse ordering: 
k, k — 1, k — 2, ... ,2,1. We will use these orders on sets [ki], . . . , [kd]. It will 
always be clear from the context which set we consider. 

Let us denote by numj/ the ordinal number of / € [k] w.r.t. the order <j. 

To define our function we need to define a specific order on the set 
K = [ki] X ... X [kd]- The construction below is essentially the same as 
in [12]. Our order will be similar to the lexicographic one, that is to compare 
two tuples from K we will compare their components one by one until we 
find the difference. But as opposed to the lexicographic order, where each 
component of the tuples is compared w.r.t. the same ordering, in our order 
of the tuples components might be compared w.r.t. different orderings. 
Moreover, the ordering in which we compare the current component depends 
not only on the ordinal number of the component but on the values of 
previous components of the tuples. 

Formally, suppose we want to compare tuples a = {ai, . . . , ad) € K and 
/3 = (/3i, . . . , f3d) € K. First we compare ai and /3i w.r.t. the ordering <i. 
If they are not equal then we have already compared the tuples: the larger 
the first component is the larger the tuple is. If they are equal we proceed 
to the second components. To compare them we use the following recursive 
rule to choose the next order. 
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2 

12 3 ... /ci "1 

Case d = 2 



Assume that the order to compare the Zth components of the tuples 
is already determined and it happens that ai = The order to compare 
(/ + l)st components is determined by the ordinal number of ai (which 
coincides with f3i by the assumption) w.r.t. the order Namely, 

ii+i = nuuii^ai (mod 2). (2) 

In other words, we compare the (/ + l)st coordinates w.r.t. the order <o if 
ai has even ordinal number w.r.t. the order <jj and we compare (/ + l)st 
coordinates w.r.t. the order <i otherwise. 

To say it the other way, we associate with coordinates of any tuple 
a = {ai, . . . ,ad) G K orders <j^,...,<i^ according to the rule ([2]). We 
use these orders to compare coordinates of a with coordinates of other tu- 
ples. Note that for two tuples a = (ai, . . . , ad) and /3 = (/3i, . . . , /3rf) the 
orders corresponding to their components coincides until we meet the first 
difference. After the first difference the orders corresponding to components 
might be different in a and /3 but we do not need to compare coordinates any 
further. Let us denote by nuuia^iai the ordinal number of the Ith component 
of a w.r.t. the corresponding order. 

It is not hard to describe the order we constructed in the case d = 2. 
Let us represent pairs (01,02) by points on the plane (see Figure). 
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The first component ai is associated with the horizontal axis, and the 
second component 02 - with the vertical axis. The arrows indicate the 
direction from smaller pairs to the larger ones. 

In the case d > 2 the constructed order is not so easy to describe. How- 
ever, as our proof goes by induction, we always consider only two consecutive 
coordinates of the tuple. Thus we will be in the situation that is very sim- 
ilar to the case d = 2. The only difference is that the order on the first 
coordinate might be different from <i. 

Now we can define our function. 

Definition 4. For given (x, y) = (x^, x^, . . . , x'^, y^-ilj^ , • • • , y'^), where re* = 
(4,...,4.),y^ = {y\,...,yl) G {0,1}*^' let a = e -f^ be the 

largest tuple w.r.t. the introduced order such that nf=i(^ai ~ Vlci) 0- 
Then let 

d 

f{x\ ...,x\y\...,y'') = sgnJlK - vU)- 

1=1 

If there is no such a let f{x^, . . . ,x'^,y^, . . . , y'^) = 1. 

Note that if d = 1 our function is exactly the GT function. 

3 2"(2" 4) Lo^er Bound 

First we note that our function is computable by a degree d threshold gate. 
Lemma 5. deg_|_(/) ^ d. 
Proof. Let 

a\«2,...,al^l 

be the list of all elements of K written w.r.t. the introduced order. For any 
= (a{, . . . , oP^ let tj be the product 

i=i ' 

Note that for any j and any input {x,y) we have \tj\ ^ 1. 
Consider the polynomial 

1^1 

p{x,y) = J2^Hj. 
3=1 



7 



We claim that it sign-represents our function. 

Indeed, let us fix some input {x,y). If for this input all tj are zeros 
then f{x,y) = 1 and p{x,y) = 0. Otherwise, let be maximal w.r.t. the 
introduced order such that tj 7^ 0. Then all ti for i > j are zeros, and the 
coefficients of p are chosen in such way that the contribution of tj is greater 
than the sum of contributions of all ti for i < j. Thus the sign of p{x,y) is 
the same as the sign of tj. □ 

Now we proceed to the main result of this section. 

Theorem 6. Let ki ^ 2 be even for i < d and kd ^ 3. Then 

deg±(/) = d 

W{f,d)^2 

Remark. We can state analogous theorem for arbitrary ki ^ 2 for i < d and 
not only for even. However, with this assumption the proof and the bound 
are cleaner and at the same time the theorem still gives 2^^^"^*) bound. 

First we give some corollaries of Theorem [6] and then proceed to the 
proof. 

If we let ki = 2 for i < d and A;^ = 3 in Theorem [6] we get 
More generally. 

Corollary 7. For all n and all d ^ there exists (explicit) function f of 
n variables such that 

Proof. Let ki = 2[^^^J for i < d, k^ = 2[^^^J+1, consider the function from 
Theorem [6] on first 2^^^^ ki variables and leave other variables inessential. 

□ 

Analogously we can prove other bounds. For example we can prove the 
following bound with a simpler formulation 

W(/,d)^2(2L^J)^-'^. 
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3.1 Proof of Theorem [6] 

Let us consider an arbitrary threshold gate p for / of degree at most d. That 
is, for any x,y ^ {0, 1}" we have 

f{x,y) = sgn{p{x,y)). 

It will be convenient for us to work in variables 

So after substituting 

x] = ^+v})/2, y) = ^-v})/2 

and multiplying the polynomial by 2*^ to make the coefficients integer we 
obtain the polynomial p' in variables u*-, Vj that sign-represents /. That is 

f{x, y) = sgn(p'(a; -y,x + y)). 

It is easy to see that the weight of the new polynomial is almost the same 
as the weight of p (compared to the value of our bound). Namely, we have 
the following bound. 

Lemma 8. W{p') ^ 2'^W{p). 

Proof. Consider one monomial g of p of degree / ^ d. We can think of 
the transformation above as of substituting each variable by the sum (or 
difference) of two variables and then multiplying everything by 2'^~K After 
opening the brackets we will have 2' new monomials each of weight 2'^~\ so 
the overall weight of the new monomials coming from the monomial g is 2*^ 
times the coefficient of g. Since this happens for all monomials, the upper 
bound on the weight of the polynomial p' as stated follows. □ 

Remark. A similar bound holds in the other direction too, but we do not 
need it. 

Now we have to prove that 

W{p') ^ 2('='^-2)^tlfc^ 

First we will prove that we can assume that p' has a nice structure. 
Lemmas similar to the next one appeared in [H [12] (see [13] for a more 
general version). 
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Lemma 9. // we substitute by all coefficients of the monomials of p' in 
which variables from one of the groups u^, . . . ,u'^ are not presented, the 
resulting polynomial q will also sign-represent f . 

Proof. Let / be the set of inputs (x, y) to / such that x* 7^ for all i = 
1, . . . ,d. This condition implies that for each such input there is an a = 
(ai, . . . , ad) & K such that 

d 
i=l 

This means that the value of the function is determined by the sign of this 
product for the largest such a. 

Consider an arbitrary input (x, y) from / and consider an arbitrary i G 
[d\. Let us permute the variables with the variables y*. It is not hard to 
see that after such permutation the function / changes sign, and so p' also 
should change sign. 

Now let us sec what happens with p'{u,v). Note, that after this per- 
mutation all variables u and v remains the same except which changes 
sign. Let us denote by A the sum of all monomials of p' which contain even 
number of variables from and by B the sum of all monomials of p' which 
contain odd number of variables from u^, so 

p'{u, v) = A{u, v) + B{u, v). 

Note that after our permutation of variables A remains the same and B 
changes sign. Since p' changes sign we have that the absolute value of B is 
greater then the absolute value of A. Wc proved that this happens for any 
input from I, so the sign of / is determined by the sign of B for all such 
inputs. So we can erase all monomials from A and the resulting polynomial 
will still sign-represent / for all inputs from /. 

Repeating this argument for all i S [d] we obtain a threshold gate q for 
/ such that each monomial of q contains odd number of variables from each 
of the groups u^,. . . ,u'^. But the degree of q is at most d, so it is only 
possible that each monomial of q contains one variable from each of the 
groups u^, . . . ,u'^ (and no variables from v^, . . . , v'^). 

We have proved that the new polynomial q sign-represents / correctly 
for all inputs from /. Now note, that for any input from {0, 1}" \ I there is 
an i G [d] such that = y*, or equivalently, all variables from are zeros. 
By the definition the value of the function / on such input is 1 and the value 
of q is 0, so q sign-represents / for these inputs also. □ 



10 



As a byproduct of the proof of this lemma we have the fohowing corohary. 

Corollary 10. deg_|_(/) = d. 

Since W{p') ^ W{q) it is enough to prove that 

Wiq) ^ 2('=<'-2)nt/'=«. 

Now we need a lemma concerning degree 1 threshold gates for GT. The 
argument is quite standard (see [lOl [TT])- 

Lemma 11. Let p = Yli=i'^i'^i ^ degree 1 threshold gate for GT(a;,y) 
where x,y £ {0, 1}'^. Then for j ^ 2 

Wj ^ > (4) 

and 

Wj ^ Wj-i. (5) 
Proof. We will actually prove that for each j Wj > and for each j ^ 2 

i-i 

^ X^iWj- (6) 

1=1 

The inequality ^ follows from this immediately and the inequality (jH) 
is easy to prove by induction. 

To prove the inequality ([S]) we consider the specific input 

= (^T~^"^,1,0...,0). 

It is easy to see that on this input the function GT is 1, and this means that 
p{u) ^ 0. It is easy to see that the inequality ([6]) follows. 

To prove that wj > just let Uj = —1 and Uj = for i ^ j- GT is zero 
for such input, so p{u) < 0. □ 

It will be convenient for us to consider two variants of the GT function: 
we denote by GTi the usual GT function and by GTq the analogous function 
but now on the reversed input. That is, GTo(x, y) = 1 if and only if x ^ y, 
where x = (xi, . . . ,Xk),y = (yi, • • • ,yk) are considered as binary represen- 
tations of integer numbers where the most significant bits are xi,yi. It is 
easy to see that if 

k 

P = ^ WiUi 
i=l 
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is a threshold gate for GTq then we have 

Wj-l ^ Wj 

and 

Wn-j+l > 2^''^Wn > 0, 

where j = 2, . . . n. 

Now we can prove the main lemma. 

Lemma 12. For all I ^ d if a K is such that nuuia^iai = 1 for all i ^ I 
and /3 = {ai , . . . ,ki — ai + 1, . . . ,ad)- Then 

ti;;3^^.„2('=<*-2)^t7^^ 

The idea is the following: we fix variables in all groups u^, . . . ,u'^ except 
one. Then the function / becomes essentially the GT function and we can 
apply Lemma [TTJ Repeating this trick we can accumulate the large factor 
due to the specific construction of our order. More specifically the proof 
goes by induction on decreasing /. For the base of induction I = d we hx 
all variables except u'^ and applying inequality (j4]) immediately obtain the 
desired result. For the induction step we first apply the induction hypothesis 
to / + 1 and then apply inequality ^ to the Ith coordinate. Then we can 
again apply induction hypothesis to ^ + 1 and so forth. In this way we 
can apply induction hypothesis ki times and obtain the desired result. We 
proceed to the detailed proof. 

Proof. The proof goes by induction on decreasing /. 

The base of induction / = d. We fix all variables n* except u'^ in the 
following way. For any i let u^^. = 1 and tx*- = for all j ^ ai. 

Now we have a function in variables u'^ and it is not hard to see that this 
function coincides with either GTq, or GTi. Applying the inequality ([4]) (or 
the corresponding inequality for GTq) for j = we obtain 

Induction step. To show the lemma for I < d we repeat several times 
the following procedure consisting of two steps. After ith application of the 
procedure we will get a tuple a* such that mim^i ial = i + 1 and mim^i ja'^ = 
1 for all j > I. To unify the notation we denote q'^ = a. 
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During the procedure we will not change values of the first I — 1 coordi- 
nates. This means that for all tuples we consider, the order corresponding 
to the Zth coordinate is the same. Let us assume without loss of generality 
that this order is <i. This in particular means that ai = 1. 

We also will not change coordinates 0^+2 , • • • , Od- Note, that this means 
that if num^i i_^_ial^^ is odd (as in the beginning) then all 

num^,^;+2a«+2> • • • > num„, ^^aj^ 

are equal to 1. 

Step 1. We apply the induction hypothesis for the coordinate l + l. We 
have that for 5*+^ = (ai, . . . , a^, ki+i — al_^_^ + 1, . . . , ad) 

Note that now the ordinal number of / + 1st coordinate (w.r.t. the corre- 
sponding order) is h^i. 

Step 2. We fix all variables except in the following way: for any 
j let itli+i = 1 and Um = for all m ^ S*"^"^. Now we have a function 

in variables and it is not hard to see that this function coincides with 
GTi (this happens because we agreed that the order corresponding to the 
/th component is <i, if it were <o we would have GTq here). We apply 
inequality ^ to the coordinate /. After that we get 

= (qi, . . . , a/ + i + 1, ki+i - 0;+^ + 1, . . . , q^) 

such that 

Due to the rule ([2]) defining the order on the next component of the tuple 
we have that the order on the I + 1st component changes. This means that 
the ordinal number of the / + 1st coordinate w.r.t. the corresponding order 
is again 1. Prom this we have, as we stated above, that for all j ^ l + l it is 
true that num„i+i ^a*^^ = 1. So now we are again in the position to apply 
Step 1. 

We repeat these two steps until the /th coordinate of 5* reaches ki (in the 
end we repeat Step 1, we are unable to repeat Step 2 since the /th coordinate 
is already ki and can not be increased, so in the end we get 5* for suitable 
i). Since the /th coordinate increases by 1 at each iteration (on Step 2) we 
can repeat Step 2 ki — 1 times and Step 1 ki times. Thus in the end we get 
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the vector a ' and it is easy to see that 

Let us check that /? = a^^ . It is easy to see that during this process only 
coordinates I and / + 1 of a changes. In the end of the process coordinate / 
has the number ki w.r.t. the order corresponding to the Ith coordinate of a. 
This means that a'j^' = ki — ai + 1 = fii. Note also that from this and from 
the evenness of ki we have that ai and a^^ define different orders on the next 
coordinates. Now let us see what happens with the coordinate / + 1. After 
the process the ordinal number of ol\\^i w.r.t. the order corresponding to 
the [l + l)st component of a^^ is equal to /c^+i. This means that the ordinal 
number of o^\_i w.r.t. the other ordering is 1. Since orders corresponding 
to the {I + l)st coordinates of a and 5*"'' are different we have that o^^^i = 
a^+i = (recall that numQ^/+ia/+i = 1). □ 

It is easy to prove Theorem [6] now. Applying Lemma [T2] with / = 1 we 

Now, it is easy to see that > (just substitute = — 1, nj^. = 1 for all 
i ^ I and n*- = for all i and all j ^ a-j). We conclude that I and 



4 Improved Lower Bound 

In this section we improve the argument of the previous sections to obtain 
the better lower bound. More precisely we prove 

We will work with Boolean variables {—1, +1}, so we change the defini- 
tion of sgn-function: sgn(x) = 1 if x ^ and sgn(x) = — 1 otherwise. 

The idea is to use another function instead of GT as a building block 
in our construction. Indeed, we may try to carry out the proof if we use 
any function /: { — — )• {—1,+!} which can be defined in the fol- 
lowing terms: consider uniform linear forms Li{x), . . . , Lk(x) and for any 
X € {— let f{x) be the sign of Li{x), where i is the maximal index 
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such that Li{x) is nonzero. Here uniformity of hnear forms is required to 
make the symmetry argument of Lemma [9] work. Note also that we can use 
different functions on different coordinates and in fact there is sense in it 
since the last coordinate plays very different role than the other coordinates. 
Actually, we have already used different functions in different coordinates 
when we choose numbers of variables ki to be different for different i. 

Note that the function GT in coordinates 1, . . . , d — 1 is not very econom- 
ical. Indeed, we use 2k variables and get only k iterations in one coordinate 
in the proof of Lemma [12] (that is, ki applications of Step 1 in the proof of 
this lemma). So we can have about n/2k coordinates with k iterations each, 
so we have bound of approximately 2 ^ '. If with some other function 
we can have more iterations with less variables, we will be able to improve 
the bound. 

We prove that for any k ^ 3 there is a function which with the use of k 
variables allows us to make k — 1 iterations. 

Definition 13. For x = {xi, . . . , x^) G {—1, +1}'^ let g{xi, . . . , Xk) be equal 
to —Xk if bits xi, . . . ,Xk are not all equal and let it be Xk if they are all 
equal. 

This function can be easily represented by a linear threshold gate: 

k-l 

g{xi, ...,Xk) = sgn(^ Xi - {k - 2)xk). 

i=l 

But to make g to be suitable as a building block in our combinatorial 
construction we have to express it in terms of linear forms Li{x), . . . , Lk{x) 
(otherwise we are not able to define our ordering). Consider k linear form: 
Li{x) = Xi — Xj+i {Li will play a role of in the previous proof) for i = 
1, . . . , /c — 1 and Lq{x) = xi + Xk- The alternative (equivalent) definition 
of g is that g{x) is equal to the sign of the last nonzero in the sequence 
Lq{x),Li{x),L2{x), . . . ,Lk-i{x). Note that now it is more convenient for 
us to start numeration of Li from 0. The reason for this is that the actual 
benefit we will get only from linear forms Li, . . . ,Lk~i- The form Lq is 
needed only for technical reasons (the same role was previously played by 
variables v^). 

Note that in the proof of weaker bound we needed actually not one base 
function, but two of them. They were very similar though: GTq and GTi. 
In case of stronger bound two functions will differ more substantially. Again, 
gi is just the function g we defined above. As for go we let goix) be xi if not 
all bits of the input are equal and go{x) = —xi if all bits of input are equal. 
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That is now we not only reverse the order of variables but also multiply the 
value of the function by —1. Note that go{x) is equal to the sign of the last 
nonzero in the sequence —Lq{x), Lk~i{x), Lk-2{x), ■ ■ ■ ,Li{x). 

Now we can apply the previous proof scheme with the functions gi and 
go on first d—1 components and with GT on the last component. We denote 
the new function by / again. Below we state what changes in the proof. 

For the new function the construction of the ordering is the same except 
that we use different orderings <q and <i on the first d — 1 coordinates, 
namely we let <[ to be 0, 1, 2, A: — 2, A; — 1 and <q to be 0, /c — 1, A: — 
2, . . . , 2, 1, that is, is always the smallest element. The orderings on the 
last coordinate remains the same as before (as well as the rule ([2]) defining 
the ordering on each next coordinate). 

In the Definition U] we now have only variables x^, x^, . . . , x'^, y*^ and we 

let 

fix) = sgn((-l)^^+-+^-^L„,(xi)L,,(x2) . . . K^_^ix''-'){xi^ - y^J) 

for the largest a for which the expression is nonzero, where Cj = 1 if = 
and the order corresponding to the i-th coordinate of a is <q and Cj = 
otherwise. If there is no such a (which can happen only if x'^ = y'^) we let 
/(x) to be 1. Note that the number of variables n = YliZi ki + ^k^ is almost 
twice less than before. 

The theorem we prove has the following form. 

Theorem 14. Let ki ^ 3 be odd for i < d and kd ^ 3. Then 

deg±(/) = d 

W{f,d) ^ 2'^'^''~^^nti {fci-i)-diogTi 

The proof of the theorem follows the same lines with minor changes. For 
the sake of completeness we present the details. 

Let us consider an arbitrary threshold gate p for / of degree at most d. 
That is, for any x^, . . . , x*^""^, x'', y'^ we have 

f{x\ . . . , x'^-\ x^ = sgn(p(x\ . . . , x'^-\ x^ y'^)). 

It will be convenient for us to work in variables 

= Lj(x*) 

for i = 1, . . . , d — 1 and 
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So after substituting 

x) = (4 - ui - . . . - + 4 + . . . + 4-i)/2 
for i = 1, . . . , d — 1 and 

and multiplying the polynomial by 2'^ to make the coefficients integer we 
obtain the polynomial p' in variables Uj, Vj that sign-represents /. That is 

f{x\ . . .,x^-\x'',y^) = sgn{p'iu\ . . . , u''-\ u'' , v'')) . 

It is easy to see that the weight of the new polynomial is almost the same 
as the weight of p (compared to the value of our bound). Namely, we have 
the following bound. 

Lemma 15. W{p') ^ n'^W{p). 

Proof. Consider one monomial g of p of degree I ^ d. We can think of 
the transformation above as of substituting each variable by the sum of 
at most maxj fcj ^ n variables and then multiplying everything by 2'^~K 
After opening the brackets we will have at most ra' new monomials each of 
weight 2*^"', so the overall weight of the new monomials coming from the 
monomial g is at most n'^ times the coefficient of g. Since this happens for 
all monomials, the upper bound on the weight of the polynomial p' as stated 
follows. □ 

Next we prove that we can assume that p' has a nice structure. 

Lemma 16. // we substitute by all coefficients of the monomials of p' 

in which variables from one of the groups v} , . . . ,u'^ are not presented, the 
resulting polynomial q will also sign-represent f . 

Proof. Let I be the set of inputs y) to / such that x'^ ^ y'^. This 
condition implies that for each such input there is an a = (ai, . . . ,a(i) G 
K = [ki - l]o X . . . X [kd-i - l]o X [kd] such that 

where by [A;]o we denote the set {0, . . . k}. This means that the value of the 
function is determined by the sign of this product for the largest such a. 



17 



Consider an arbitrary input {x,y) from /. Let us first permute the 
variables x'^ with the variables y'''. It is not hard to see that after such 
permutation the function / changes sign, and so p' also should change sign. 

Now let us see what happens with p'{u^, ■ ■ ■ , u'^-, v*^). Note, that after this 
permutation all variables u and v'^ remains the same except u'^ which changes 
sign. Let us denote by A the sum of all monomials of p' which contain even 
number of variables from u"^ and by B the sum of all monomials of p' which 
contain odd number of variables from u'^, so 

p\u, v'^) = A{u, v'^) + B{u, v'^). 

Note that after our permutation of variables A remains the same and B 
changes sign. Since p' changes sign wc have that the absolute value of B is 
greater then the absolute value of A. We proved that this happens for any 
input from I, so the sign of / is determined by the sign of B for all such 
inputs. So we can erase all monomials from A and the resulting polynomial 
will still sign-represent / for all inputs from /. 

Now consider an arbitrary i G [d — 1] . Let us substitute the vector a;* by 
the vector — x*. Again, it is not hard to see that after such substitution the 
function / changes sign, and so p' also should change sign. 

Let us see what happens with p'{u^, . . . ,u'^,v'^). Note, that after this 
substitution all variables u and v'^ remains the same except ti* which changes 
sign. Let us denote by A the sum of all monomials of p' which contain even 
number of variables from and by B the sum of all monomials of p' which 
contain odd number of variables from u*, so 

p'{u, v'^) = A{u, v'^) + B{u, v'^). 

After our substitution A remains the same and B changes sign. Again, 
we can erase all monomials from A and the resulting polynomial will still 
sign-represent / for all inputs from /. 

Repeating this argument for all i € [d — 1] we obtain a threshold gate q 
for / such that each monomial of q contains odd number of variables from 
each of the groups u^, . . . But the degree of q is at most d, so it is 
only possible that each monomial of q contains one variable from each of the 
groups u^, . . . , n'^ (and no variables from v'^). 

We have proved that the new polynomial q sign-represents / correctly 
for all inputs from I. Now note, that for any input from {— 1, 1}" \/ we have 

_ yd^ or equivalently, all variables from are zeros. By the definition 
the value of the function / on such input is 1 and the value of q is 0, so q 
sign-represents / for these inputs also. □ 
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As a byproduct of the proof of this lemma we have the fohowing corohary. 
Corollary 17. deg_|_(/) = d. 

Since W{p') ^ W{q) it is enough to prove that 
W{q) ^ 2('='*-2)nti(fe.-i). 

We present now the analog of Lemma [TTJ 

Lemma 18. Let p = "^{Zq ''^iUi be a degree 1 threshold gate for gi{x) where 
X E {—1, Then for j = 0, 1, . . . , /c — 1 we have Wj > and for 

j = 2, . . . ,k — 1 we have wj > wj-i. 

For the function go analogous statement is true. 

Lemma 19. Let p = "^^Zq WiUi be a degree 1 threshold gate for go{x) where 
X G {—1, +1}'^. Then we have wq < 0, for j = 1, . . . ,k — 1 we have Wj > 
and for j = 2, . . . ,k — 1 we have wj-i > Wj. 

Proof of Lemma \18[ To show the first part of the lemma for j ^ let xi = 
. . . = Xj = —1 and Xj+i = ... = Xfc = 1. Then gi{x) = — 1 and thus 
p{x) = -2wj < 0. 

For j = let xi = ... = Xk = —1. Then again gi{x) = —1 and 
p{x) = —2wo < 0. 

To show the second part let Xj = —1 and xi = 1 for I ^ j. Then we have 
gi{x) = —1 and p{x) = 2{wj-i — Wj + wq) < 0. And thus wj > wj-i + > 

Wj-l. □ 

Proof of LemmaUM For j ^ letting xi = ... = xj = —1, Xj+i = . . . = 
Xfc = 1 we have —Wj < 0. Letting xi = . . . = = 1 we have wq < 
(note that we have —Lq in the sequence defining go). For j = 2, . . . ,k — 1 
letting Xj = 1 and = — 1 for / / j we have —Wj-i + Wj — wq < and thus 
Wj-i > Wj since wq is negative. □ 

The analog of Lemma [12] is very similar to the previous version, but 
becomes a little bit clumsy since we distinguish cases of I = d and / < d. 

Lemma 20. For all I ^ d if a K is such that nuuia^iai = 1 for all i ^ I 
and f3 = (ai, . . . ,ki — ai + 6i^d, ■ ■ ■ , where 5i^d is a Kronecker delta (that 
is 6ij = 1 if i = j and 5ij = otherwise) then 
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Concerning the proof of the lemma, the base of induction remains com- 
pletely the same (note, that the statement is the same also). As for the 
induction step, it also remains the same but now we can apply the induc- 
tion hypothesis ki — 1 times instead of ki times in the previous proof. For 
the sake of completeness we present the proof. 

Proof. This proof repeats the proof of Lemma [12] almost literally. 
The proof goes by induction on decreasing 

The base of induction / = d. We fix all variables n* except u"^ in the 
following way. For any i let u^. = 1 and = for all j ^ Oi (see the proof 
of Lemma [18] on how to do this). 

Now we have a function in variables u'^ and it is not hard to see that this 
function coincides with either GTq, or GTi. Applying the inequality (or 
the corresponding inequality for GTq) for j = k^ we obtain 

Induction step. To show the lemma for I < d we repeat several times 
the following procedure consisting of two steps. After ith application of the 
procedure we will get a tuple a* such that num^,! = i + 1 and num^i ja* = 
1 for all j > I. To unify the notation we denote = a. 

During the procedure we will not change values of the first I — 1 coordi- 
nates. This means that for all tuples we consider, the order corresponding 
to the Ith coordinate is the same. Let us assume without loss of generality 
that this order is <'^. This in particular means that ai = 1. 

We also will not change coordinates ai+2, • • • , Od- Note, that this means 
that if num^i i_^_ial_^_^ is odd (as in the beginning) then all 

num^.^;+2aI+2> • • • > num^.^^a'^ 

are equal to 1. 

Step 1. We apply the induction hypothesis for the coordinate l + l. We 
have that for 5*+^ = (oi, . . . , a^, k+i - q^_^^ + . . . , a^) 

«;5.+i^u;„.2('='^-2)nt7+.('=-i). 

Note that now the ordinal number of / -1- 1st coordinate (w.r.t. the corre- 
sponding order) is ki+i - 1 + 6i+i^d- 

Step 2. We fix all variables except in the following way: for any 
j let tt~,+i = 1 and Um = for all m / S*^^. Now we have a function 
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in variables u and it is not hard to see that this function coincides with 
gi (this happens because we agreed that the order corresponding to the 
/th component is <'^, if it were <o we would have go here). We apply 
inequality from Lemma [18] to the coordinate I. After that we get a*"*"^ = 
(oi, . . . , + i + 1, ki+i - qJ_^^ + 6i+i^d, • • • , "d) such that 

Due to the rule ^ defining the order on the next component of the tuple 
we have that the order on the / + 1st component changes. This means that 
the ordinal number of the / + 1st coordinate w.r.t. the corresponding order 
is again 1. From this we have, as we stated above, that for all j ^ / + 1 it is 
true that num„i+i ja*^^ = 1. So now we are again in the position to apply 
Step 1. 

We repeat these two steps until the /th coordinate of 5* reaches ki — 1 
(in the end we repeat Step 1, we are unable to repeat Step 2 since the Zth 
coordinate is already ki — 1 and can not be increased, so in the end we get 
5* for suitable i). Since the /th coordinate increases by 1 at each iteration 
(on Step 2) we can repeat Step 2 ki — 2 times and Step 1 ki — 1 times. Thus 
in the end we get the vector 5'^'"^ and it is easy to see that 

^5^.- > ^„ n 2(^''^-2)nt-.Vi('=-i) = «;,2(^''^-2)nf-{^.-i). 

Let us check that /3 = a^^~^. It is easy to see that during this process 
only coordinates / and /+1 of a changes. In the end of the process coordinate 
/ has the number ki — 1 w.r.t. the order corresponding to the /th coordinate 
of a. This means that 5|^'~^ = ki — ai = Pi- Note also that from this and 
from the oddness of ki we have that a/ and 5j^'~^ define different orders on 
the next coordinates. Now let us see what happens with the coordinate / + 1. 
After the process the ordinal number of 5|^j^^ w.r.t. the order corresponding 
to the (/ + l)st component of a^^~^ is equal to fej+i — 1 + 5;+!,^. This means 
that the ordinal number of 5|^^^^ w.r.t. the other ordering is 1. Since orders 
corresponding to the (/ + l)st coordinates of a and a^'~^ are different we 
have that af^^ = a;+i = (recall that numa^^+iaz+i = 1). □ 

To conclude the proof of our lower bound we have to choose the values 
of ki to maximize the lower bound we have. It is not hard to see that the 
optimal way is to take k^ = 3 as before and to take ki = k2 = ■ ■ ■ = k^-i, let 
us denote the value of them by k. Then the exponent of our bound will be 
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about {k — 1)"/*^. Simple analysis shows that the maximum (over integers) 
is attained when k = 5. Thus we have a lower bound of 2^^*" ^-n^ 

To prove the first part of Theorem [2] we can just let m = ^en and 
consider the function from the previous paragraph with m variables. Then 
we have lower bound 2^^^^™''^^) for degree m/5 threshold gates and applying 
the observation preceding Theorem [2] we get the desired bound. For the 
second part of the theorem let m = |(1 + e) logn. 

Finally we note that the result of [12] can also be reproved by the same 
argument and with better constants if we use Hastad's function in the last 
coordinate and g function in other coordinates. 
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